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STATEMENT OF THE CASE 
Appellants appeal under 35 U.S.C. § 134 from the Examiner's 
rejection of claims 31 to 38. We have jurisdiction under 35 U.S.C. § 6(b). 
We AFFIRM. 

Appellants' invention relates to a method and apparatus for 

automatically identifying a location in a video frame, determining a color 

value of the location, and associating an event with the color value (Spec. 14 

and 15, Fig. 1). Claims 31 and 37 follow and represent the subject matter 

claimed on appeal: 

31. A method for automatically identifying an object in a 
plurality of video frames and associating the object with an event 
comprising: 

determining a location in one of said video frames where an 
action by a pointing device has occurred defining a selected location; 

determining a color value of said selected location; and 

automatically associating an event with said color value of said 
selected location in said one video frame and automatically 
associating events with said color value in succeeding video frames. 

37. A method for automatically identifying an object in a 
plurality of video frames and associating the object with an event 
comprising: 

determining the coordinates along one edge of an object in one 
of said video frames selected by a pointing device defining a selected 
object; 

determining the color values along said edge of said selected 
object; and 
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automatically associating an event with said color values along 
said edge in said one video frame and automatically associating events 
with said color values in succeeding video frames associated with the 
color values. 

REJECTIONS 

The Examiner relies on the following prior art references to show 
unpatentability: 

Rangan US 6,198,833 Bl Mar. 6, 2001 

Isadore-Barreca US 6,205,231 Bl Mar. 20, 2001 

The Examiner rejects the claims as follows: 

1. Claims 31 to 36 are rejected under 35 U.S.C. § 102(b) 2 as being 
anticipated by Rangan. 

2. Claims 37 and 38 are rejected under 35 U.S.C. § 103(a) as 
being unpatentable over Rangan and Isadore-Barreca. 

Rather than repeat the arguments of Appellants or the Examiner, we 
refer to the Appeal Brief filed on October 11, 2006, the Examiner's Answer 
mailed on October 27, 2006, and the Reply Brief filed on September 28, 
2006 for their respective details. Appellants argue two groups of claims. 
Appellants collectively argue claims 31 to 36 and state that these claims 



2 Although the Examiner indicated that these claims were rejected under 

§ 102(b) in the Answer (Ans. 3), the Examiner nonetheless indicates that the 

claims were rejected under § 102(e) in the Final Rejection. Compare Ans. 3 

with Fin. Rej. 4. In any event, Rangan qualifies as prior art under 

§ 102(e). Nevertheless, we deem this error in the Answer harmless as it 

does not affect our decision regarding the merits of the anticipation 

rejection. 
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stand or fall together (App. Br. 7). 3 Similarly, Appellants collectively argue 
claims 37 and 38 (App. Br. 9 and 10; Reply Br. 3 to 5). Arguments which 
Appellants could have made but did not make in their Briefs have not been 
considered and are deemed to be waived. See 37 C.F.R. § 1.37(c) (1) (vii). 
Claims 31 and 37, which are set forth above, represent the invention claimed 
on appeal. Claims 32 to 36 stand or fall with claim 31. Claim 38 stands or 
falls with claim 37. 

ISSUES 

The Examiner identifies where all limitations in claims 31 to 36 on 
appeal are taught by Rangan. The Examiner also identifies where all 
limitations in claims 37 and 38 on appeal are taught by Rangan and Isadore- 
Barreca. The Examiner provides reasons for one of ordinary skill in the art 
to combine the teachings of Rangan and Isadore-Barreca. Appellants argue 
that Rangan and/or Isadore-Barreca do not teach the limitations in the claims 
on appeal. Appellants also argue that there is no motivation to combine the 
teachings of Rangan and/or Isadore-Barreca. 

From the above contentions, we frame the issues on appeal as follows: 

1 . Have Appellants shown that the Examiner erred by not 
establishing that all limitations in claims 31 to 36 on appeal are 
shown in Rangan within the meaning of 35 U.S.C. § 102? 

2. Have Appellants shown that the Examiner erred by not 
establishing that all limitations in claims 37 and 38 on appeal 

3 The Briefs were filed in the USPTO by facsimile. The copy of the Briefs 
in the USPTO record only display page numbers inserted during the 
facsimile transmission. We use the page numbers at the bottom left of the 
pages for our references. 
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are shown within the combined teaching of Rangan and 
Isadore-Barreca within the meaning of 35 U.S.C. § 103(a)? 
3. Have Appellants shown that the Examiner erred by not 
providing a sufficient reason to combine the teachings of 
Rangan and Isadore-Barreca within the meaning of 35 U.S.C. § 
103(a)? 



FINDINGS OF FACT 
The record supports the following findings of fact (FF) by a 
preponderance of the evidence. 

1. Appellants state that the invention claimed on appeal determines 
an item of interest in an initial frame, and then automatically 
locates the selected item in the remaining frames, and these items 
are then tagged and linked to an event, such as a URL (App. Br. 6 
and 7). 

2. Appellants admit that Rangan tracks an area (e.g., a rectangular 
wire frame) "having a fixed size around the location where the 
pointing device was exercised by comparing the color of 
individual pixels in the fixed wire frame from one frame to the 
next" (App. Br. 8). 

3. Rangan teaches a graphical user interface (GUI) usable by an 
editor for selecting and centering a tracking element on a moving 
entity to be tracked in a first frame of video, and for initiating a 
tracking process (col. 3, 11. 29 to 64). The tracking element 29 is 
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placed over an image to be tracked and activated by a mouse click 
or other command action (col. 7, 11. 34 to 52). 

4. Rangan discloses a step of determining a color value with a 
selected location and automatically associating an event. For 
example, "the color (e.g. RGB) values of each target pixel are 
used as a base comparison signature for tracking element 29 
during frame by frame motion (internal frame rate)" (col. 7, 11. 35 
to 52). 

5. Rangan also discloses initiation of "tracking of any image entity 
or entities in a video stream, after which initiation tracking may be 
automatic" (col. 4, 1. 66 to col. 5, 1. 19). An image entity is "any 
person or thing depicted in a video display, such as a player in a 
sports game, and actor in a play, a car in a car race, and so on" 
(col. 5, 11. 8 to 10). 

6. Rangan teaches automatically associating an event with the color 
value of the selected location in one video frame and 
automatically associating events with the color value in 
succeeding video frames (col. 4, 11. 20 to 32). 

7. Rangan provides an example of an initiating event in which 
tracking element 29 is placed over an image entity (i.e., a swim 
suit 4 worn by a swimmer) in an initial frame to be tracked, and a 
mouse click or some other action starts the tracking process (col. 
7, 11. 34 to 38). The color values (i.e., RGB) for target pixels in 

4 In Appellants' disclosure, the initiating event is a moving green shirt worn 
by a game show host (Spec. 15). The selecting and tracking of the green 
shirt in Appellants' disclosure is equated to the selecting and tracking of the 
swimsuit worn by the swimmer in Rangan. 
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the initially tracked image are stored in a table, and are used as a 
base comparison signature for subsequent frames tracked by 
tracking element 29 (col. 7, 11. 38 to 52; col. 8, 11. 34 to 55; col. 9, 
11. 12 to 40; col. 10, 11. 43 to 46). 

8. Isadore-Barreca, like Rangan, is concerned with the processing of 
moving video images and more particularly to tracking moving 
entities in a video presentation (col. 4, 1. 66 to col. 5, 1. 6) (F. Rej. 
9 to 13). Further, Isadore-Barreca identifies and traces an object 
within a video image (col. 5, 11. 44 to 62). 

9. In Isadore-Barreca, an object 20 is depicted upon a background 
22. The background 22 may include nondescript and generally 
undefinable portions and other items 24, which other items 24 are 
potentially definable portions of the background 22. These other 
items can be separately defined as additional objects of interest 
(Fig. 2, col. 5, 11. 36 to 44). Isadore-Barreca indicates that 
methods for identifying, deleting, or moving the object 20, or 
changing its color within the video image 14 are known in the art 
(col. 5, 11. 45 to 49). According to Isadore-Barreca, a well-known 
and widely practiced edge detection technique is known "as 
Sobel's Algorithm (discussed in detail in Digital Image Processing 
by Gonzolez and Woods, Addison- Wesley Publishing, 1992, pp. 
197-201 and 416-420)" can be used to detect an edge 26 of the 
object 20 (col. 5, 11. 49 to 55). The edge 26 is a boundary between 
two regions of an image (the two regions being the object 20 and 
the background 22) "which two regions have relatively distinct 
luminosity characteristics, although it is certainly conceivable that 
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the edge 26 [can] be defined according to color characteristics, or 
the like, in addition to or instead of by the luminosity 
characteristics" (col. 5, 11. 55 to 62). 

PRINCIPLES OF LAW 
Claim Interpretation 
The transitional term "comprising" is inclusive or "open-ended and 
does not exclude additional, unrecited elements," Mars, Inc. v. H.J. Heinz 
Co., 377 F.3d 1369, 1376 (Fed. Cir. 2004) (citations omitted). 

Anticipation 

"A claim is anticipated only if each and every element as set forth in 
the claim is found, either expressly or inherently described, in a single prior 
art reference." Verdegaal Bros., Inc. v. Union Oil Co. of California, 814 
F.2d 628, 631 (Fed. Cir. 1987). 

Obviousness 

In rejecting claims under 35 U.S.C. § 103, it is incumbent upon the 
Examiner to establish a factual basis to support the legal conclusion of 
obviousness. See In re Fine, 837 F.2d 1071, 1073 (Fed. Cir. 1988). In so 
doing, the Examiner must make the factual determinations set forth in 
Graham v. John Deere Co., 383 U.S. 1, 17 (1966). 

"Section 103 forbids issuance of a patent when 'the differences 
between the subject matter sought to be patented and the prior art are such 
that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said 
subject matter pertains.'" KSR Int'l Co. v. Teleflex Inc., 550 U.S. 398, 127 S. 
Ct. 1727, 1734 (2007). 
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The Court emphasized the need to account for common sense when 
considering whether a combination of references would have been obvious: 
"[c]ommon sense teaches, however, that familiar items may have obvious 
uses beyond their primary purposes, and in many cases a person of ordinary 
skill will be able to fit the teachings of multiple patents together like the 
pieces of a puzzle." KSR, 127 S. Ct. at 1742. The Court further explained 
that " [although common sense directs one to look with care at a patent 
application that claims as innovation the combination of two known devices 
according to their established functions, it can be important to identify a 
reason that would have prompted a person of ordinary skill in the relevant 
field to combine the elements in the way the claimed new invention does." 
KSR, 127 S. Ct. at 1741. 

ANALYSIS 

Anticipation 

We are not persuaded by Appellants' arguments of Examiner error. 
Appellants argue that Rangan does not teach the step recited in claim 3 1 
calling for "determining a location in one or more of said video frames 
where an action by a pointing device has occurred" (App. Br. 7). However, 
we agree with the Examiner's finding (Ans. 3) that Rangan teaches the use of 
a graphical user interface (GUI) for determining a location in a video frame, 
such as for electing, centering, and initiating a tracking element (FF 3 and 
7). Appellants then argue that Rangan does not teach "determining the color 
value..." step in claim 31 (App. Br. 7). We disagree. As indicated supra, 
Rangan discloses a step of determining a color value of a selected location 
(FF 3 to 7). Rangan further discloses initiation of tracking of any image 
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entity or entities in a video stream, after which initiation tracking may be 
automatic (FF 5). An image entity can be any person or thing depicted in a 
video display, such as a player in a sports game, and actor in a play, a car in 
a car race, etc. (FF 5). 

Contrary to Appellants' arguments (App. Br. 7 and 8), Rangan also 
discloses the last step in claim 31 on appeal, namely, "automatically 
associating an event" (i.e., movement of the swimmer and the swim trunks) 
with the color value of the selected location in the one video frame and 
automatically associating events with the color value in succeeding video 
frames (FF 6 and 7). 

Appellants' other arguments are directed to limitations not in the 
claims on appeal. We summarize these arguments below where limitations 
not in the claims on appeal are set forth in italics: 

1 . Rangan does not define an object based on its color, does not 
dynamically track the object based on color, and does not change 
the shape and size of the object based on color (App. Br. 8 and 
9). 

2. Rangan does not automatically associate a color with an object 
(App. Br. 8). 

3. Rangan does not require a color signature that is representative 
of a continuous color pattern (App. Br. 8). 

4. Rangan does not automatically associate an object with its color 
in a plurality of video frames (App. Br. 9). 

5. Rangan does not teach simply associating an event with a 
mathematical color property; and when an action by a pointing 
device is detected in succeeding video frame, the mathematical 



10 



Appeal 2008-2687 
Application 09/679,391 

color property of the selected location in the succeeding video 
frame is determined and compared with the mathematical color 
value that has been associated with events; and if the 
mathematical color value of the selected location in the 
succeeding video frame matches a mathematical color value 
previously associated with the event, an event is initiated (Reply 
Br. 6). 

The alleged claim limitations and combination of claim limitations set forth 
in italics above are not present in claim 3 1 or other claims on appeal. 
Therefore, the alleged limitations cannot distinguish the claimed invention 
from the prior art, and Appellants cannot establish error in the Examiner' s 
rejections based thereon. 

In addition, Appellants argue that Rangan tracks a rectangular wire 
frame, having a fixed size around the location where the pointing device was 
exercised by comparing the color of individual pixels in the fixed wire frame 
from one frame to the next (Reply Br. 5 and 6), which allegedly is different 
from the claims on appeal. However, we find no patentable distinction 
between tracking a location or rectangle wire frame as taught by Rangan and 
the invention claimed on appeal. In fact, Appellants state that the invention 
claimed on appeal determines an item of interest in an initial frame, and then 
automatically locates the selected item based on the color characteristics in 
the remaining frames (FF 1 and 2). Appellants' own disclosure states that 
"an invisible wire frame is created to describe the shape of the given colored 
object to associate with the event. But the value of the pixels defines the 
wire frame rather than actually drawing a wire frame on the screen" (Spec. 
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24 and 25). Thus, Appellants' argument is without merit since the same 
wire frame is used in the described invention. 

Appellants argue Rangan discloses or requires procedures in addition 
to those claimed on appeal, namely, a relatively computative intensive 
method (Reply Br. 5.) However, the claims on appeal by the use of the open- 
ended transitional phrase "comprising" do not exclude additional procedures 
or distinguish from the use of additional procedures in the prior art. Mars, 
Inc., 377 F.3dat 1376. 

Finally, Appellants argue that Rangan teaches away from the 
invention as it teaches the use of hyperlinks, which are based upon location 
and not color value (Reply Br. 5 and 6). We do not believe that this 
argument establishes Examiner error due in part to the fact that such 
limitations do not appear in the claims on appeal. We note that Appellants' 
Specification describes that the invention claimed on appeal can be linked to 
an event, such as a URL (FF 1). 

For the foregoing reasons, Appellants have not shown Examiner error 
in the rejection of independent claim 31 over Rangan. Accordingly, we 
affirm the 35 U.S.C. § 102 rejection of claim 31, as well as claims 32 to 36 
that fall therewith because of the lack of arguments directed thereto. 

Obviousness 

The Examiner cited Isadore-Barreca as teaching the limitations 
concerning "determining the coordinates along one edge of an object..." as 
required in claim 37 on appeal. Appellants do not challenge that Isadore- 
Barreca teaches this limitation in claim 37 on appeal (App. Br. 9 and 10). 
Instead, Appellants argue that the Examiner does not provide a reason to 
combine Rangan and Isadore-Barreca (Reply Br. 4 and 6). We do not agree. 
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The Examiner provides many reasons for combining these teachings. These 
include the fact that both Rangan and Isadore-Barreca share similar 
technological environments corresponding to the processing of moving 
video images and tracking moving entities in a video presentation (FF 8 and 
9, Fin. Rej. 9 to 13). Further, Isadore-Barreca identifies and traces an object 
within a video image (FF 8 and 9, F. Rej. 9 to 13). Accordingly, one of 
ordinary skill in the art would have found it obvious combine the teachings 
of Rangan and Isadore-Barreca. 

For the foregoing reasons, Appellants have not shown Examiner error 
in the rejection of independent claim 37 over Rangan and Isadore-Barreca. 
Accordingly, we affirm the 35 U.S.C. § 103(a) rejection of claim 37, as well 
as claim 38 that falls therewith for lack of arguments directed thereto. 

CONCLUSIONS 

Appellants have not shown that the Examiner erred by failing to 
establish that all limitations in claims 31 to 36 on appeal are shown in 
Rangan within the meaning of 35 U.S.C. § 102. 

Appellants have not shown that the Examiner erred by failing to 
establish that all limitations in claims 37 and 38 on appeal are shown within 
the combined teaching of Rangan and Isadore-Barreca within the meaning of 
35 U.S.C. § 103(a). 

Appellants have not shown that the Examiner erred by failing to 
provide a sufficient reason to combine the teachings of Rangan and Isadore- 
Barreca within the meaning of 35 U.S.C. § 103(a). 

ORDER 
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The Examiner's rejection of claims 31 to 36 under 35 U.S.C. § 102 as 
being anticipated by Rangan is affirmed. 

The Examiner's rejection of claims 37 and 38 under 35 U.S.C. 
§ 103(a) as being unpatentable over Rangan and Isadore-Barreca is affirmed. 
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No time period for taking any subsequent action in connection with 
this appeal may be extended under 37 C.F.R. § 1.136(a). 



AFFIRMED 



ELD 



KATTEN MUCHIN ROSENMAN LLP 
(C/O PATENT ADMINISTRATOR) 
2900 K STREET NW, SUITE 200 
WASHINGTON, DC 20007-5118 
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